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Competition among cooperators, defectors, and loners is studied in an evolutionary prisoner's dilemma game 
with optional participation. Loners are risk averse i.e. unwilling to participate and rather rely on small but fixed 
earnings. This results in a rock-scissors-paper type cyclic dominance of the three strategies. The players are 
located either on square lattices or random regular graphs with the same connectivity. Occasionally, every player 
reassesses its strategy by sampling the payoffs in its neighborhood. The loner strategy efficiently prevents suc- 
cessful spreading of selfish, defective behavior and avoids deadlocks in states of mutual defection. On square 
lattices, Monte Carlo simulations reveal self-organizing patterns driven by the cyclic dominance, whereas on 
random regular graphs different types of oscillatory behavior are observed: the temptation to defect determines 
whether damped, periodic or increasing oscillations occur. These results are compared to predictions by pair ap- 
proximation. Although pair approximation is incapable of distinguishing the two scenarios because of the equal 
connectivity, the average frequencies as well as the oscillations on random regular graphs are well reproduced. 
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I. INTRODUCTION 

Evolutionary prisoner's dilemma games (PDG) []lj, |2[ 
|5|] were introduced to study the emergence and maintenance 
of cooperation among selfish individuals in societies where 
strategies are either inherited or adopted through basic imi- 
tation rules. In the original PDG |^] two players simultane- 
ously decide whether to cooperate or defect. Mutual cooper- 
ation (defection) yields the highest (lowest) collective payoff 
which is shared equally. However, still higher individual pay- 
offs are achieved by defectors facing cooperators leaving the 
latter with the lowest possible payoff. Thus, defection is dom- 
inant because they fare better (or at least equal) regardless of 
the co-players decision. Consequentially, 'rational' players al- 
ways end up with the lowest collective payoff instead of the 
higher reward for mutual cooperation. 

Obviously, this result is at odds with observations in hu- 
man and animal societies. Indeed, cooperation may emerge 
under certain circumstances (see e.g. [|7| f| [| [l(| |lU [l^]). In 
well-mixed populations, i.e. with random matchings, cooper- 
ative strategies succeed provided that interactions between the 
same individuals are repeated with sufficiently high probabil- 
ity. Computer tournaments [|], |l3]] emphasized the importance 
of particularly simple strategies such as D (always defect), C 
(always cooperate), and T ("tit for tat", cooperate on the first 
move and then repeat the co-player's move). 

In spatially extended systems with limited local interac- 
tion, cooperators may thrive by forming clusters [JflJ, [14], |l5[| 
and thereby reducing exploitation by defectors. For exam- 
ple, on square lattices with C, D strategies and suitable pa- 
rameter values, clusters of cooperators typically expand along 
straight boundaries while being invaded by defectors along 
corners and irregular boundaries. The competition between 
these two invasion processes maintains persistent co-existence 
of both strategies. When tuning the model parameters, univer- 
sal phase transitions between states of homogenous defection, 
co-existence and mutual cooperation are observed [|l6[ nJU. 



Very recently the analysis is extended to diluted lattices 
and to adaptive networks JT^ ] . 

The dynamics of such spatial evolutionary PDG becomes 
increasingly complex if players can adopt three or more strate- 
gies [^0| Ell \l2\ \. In particular, the crucial role of the T 
strategy is confirmed [^3j 0]. For example, in an externally 
driven system supplying C, i.e. nourishing defectors, coop- 
erative behavior persists through the cyclic dominance of D, 
C, and T which maintains self-organizing domain structures 



In this work, we consider another PDG with three strategies 
by allowing for optional participation. Players can adopt the 
C and D strategies mentioned above or decline participation 
relying on some small but fixed source of income. Such risk 
averse players are termed loners (L). 

Voluntary participation in public goods games [ |lO| , p8| ] 
turned out to be an efficient way to prevent and avoid the 
tragedy of the commons p^]. This system has an inherent 
cyclic dominance resulting in periodic oscillations in well- 
mixed populations [^0|| and self-organizing polydomain struc- 
tures on square lattices However note that the former 
case requires group sizes larger than pairs i.e. excluding the 
PDG. Pairwise interactions invariably lead to homogenous 
states of loners. In sharp contrast, we demonstrate that co- 
operative behavior persists on square lattices producing self- 
organizing patterns whereas varying types of oscillations oc- 
cur on random regular graphs (RRG) emphasizing the impor- 
tance of topological characteristics. RRGs represent a natu- 
ral choice for social networks relevant in human behavior as 
well as economic interactions where the geometrical location 
hardly matters. 



II. THE MODELS 

We consider the evolutionary PDG in large populations 
with N players in the limit N — > oo for different geome- 
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tries: First, we briefly review the results for well-mixed popu- 
lations i.e. the mean-field approximation. Second, the players 
are arranged on a square lattice and interact only with their 
four nearest neighbors to the north, east, south and west. Fi- 
nally, the players are located on a RRG with fixed connectiv- 
ity (z — 4) excluding double or self-connectivities. There- 
fore, each player has the same number of neighbors as its 
counterpart on the square lattice. Note that locally a RRG 
is similar to a tree (or Bethe-lattice) because the average loop 
size increases with N [Q. Since boundary problems render 
Bethe-lattices unsuitable for Monte Carlo (MC) simulations, 
RRGs serve as suitable substitutes. In addition, the analysis of 
branching annihilating random walks has justified the validity 
of pair approximation on RRG p3[]. 

Each player follows one of three strategies, namely de- 
fection D, cooperation C, and loner L, denoted by a three- 
component unit vector: 

■w-0. (!).(;)■ a) 

The total payoff m(x) for the player located on site x is spec- 
ified by 

m(x) =^s T (x)M-s(x + <5), (2) 
s 

where s T (x) denotes the transpose of s(x) and the summa- 
tion runs over the four nearest neighbors (5) as defined by the 
geometry under consideration. Following [^TJ, the payoff ma- 
trix M is given by 

fOba\ 
M = I 1 a , (3) 
\ a a a ) 

where b (1 < b < 2) determines the temptation to defect. 
Loners and their co-players invariably obtain the fixed payoff 
< a < 1, i.e. they perform better than two defectors but are 
worse off than cooperative pairs. 

Randomly chosen players are allowed to reassess and mod- 
ify their strategy. For example, the player at site x adopts the 
strategy of one of its (randomly chosen) neighbors y with a 
probability 

W[s(x) - s(y)] = — — L — — T , (4) 

1 + cxp l(m(x) — m{y))/K\ 

where K introduces some noise that occasionally leads to ir- 
rational decisions. 

The imitation rule or learning mechanism (Q) leads to three 
trivial unstable fixed points, namely the homogenous C, D 
and L states. The highest average payoff (m c = 4) is achieved 
if everybody cooperates but this state is vulnerable to exploita- 
tion. For sufficiently large b the appearance of a single defec- 
tor suffices for cooperation to break down. The system then 
evolves into a homogenous D state with the lowest average 
payoff (md = 0). Loners may in turn invade this state and 
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FIG. 1: Time evolution of states in mean-field approximation for 
b = 1.5, K = 0.1 and cr = 0.3. 

thereby provide an escape hatch out of states of mutual defec- 
tion. Eventually a homogenous L state is reached with an av- 
erage payoff nij = 4cr. Note that solitary defectors are elim- 
inated by noise. Finally, the loop is closed by noting that if 
no one participates in the PDG, cooperators may thrive again. 
However, also note that this requires the presence of at least 
two cooperators. This cyclic rock-scissors-paper type domi- 
nance determines the system's dynamics over a wide parame- 
ter range. 

III. MEAN-FIELD AND PAIR APPROXIMATIONS 

In the classical mean-field approximation, i.e. in well- 
mixed populations, this system is described by the (time- 
dependent) frequencies fi(t) of the three strategies with 
fc(t) + f D (t) +/l(() = 1§| Figure Q displays the 
time evolution on a ternary phase diagram. 

Regardless of the initial configuration, the system always 
approaches the homogenous L state. This result is robust 
against changes in the parameter values. The heteroclinic cy- 
cle along the boundary of the simplex S3 reflects the cyclic 
dominance of the strategies. Mean-field approximations ne- 
glect spatial structures, i.e. they describe individuals with an 
infinite range of interactions or situations where co-players are 
randomly chosen. 

In contrast, pair approximation is capable of capturing 
some relevant effects occurring for lasting (fixed) partnership. 
The frequency of strategies is determined by two-site or pair 
configuration probabilities (for an introduction see e.g. [p4{]). 
The probability of a strategy change depends on the configu- 
ration of an eight-site cluster consisting of the randomly cho- 
sen player, one of its randomly chosen neighbors (the pair) 
and their six nearest neighbors. Configuration probabilities of 
larger clusters are assumed to be products of the correspond- 
ing pair configuration probabilities. Due to the same connec- 
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FIG. 2: Different trajectories as predicted by pair approximation: 
(a) for b — 1.03 (solid line) loners go extinct and only D and C 
survive and co-exist in a stationary state, (b) all the three strategies 
coexist for b = 1.2 (dotted line), (c) for b = 1.5 the limit cycle is 
shown (dashed line). The bullet indicates the unstable interior fixed 
point, (d) the trajectory spirals towards the boundary of the simplex 
(6 = 1.9). 



tivity z = 4 of the square lattice and the chosen RRG, the two 
structures are indistinguishable by pair approximation. 

The following systematic analysis is restricted to the case 
of a — 0.3 and K = 0.1. Figure || shows four typical tra- 
jectories for different values of the temptation to defect b. 
For very small b (1 < b < b^ = 1.0485(1)) the cluster- 
ing advantage of cooperators suffices to offset exploitation 
by defectors. Loners are unable to provide a viable alterna- 
tive and vanish. The ratio of D and C in the stationary state 
depends on the parameters b, a, and K. When increasing b 
(b ( a } < b < b^ = 1.4670(3)), a stable interior fixed point ap- 
pears, i.e. all trajectories approach this stationary value where 
all three strategies co-exist. For b > b^ periodic oscillations 
appear. Both, the amplitude and the period increase with b. 
In finite populations with b > b^ ~ 1.85 any strategy may 
go extinct due to noise and the trajectory ends in one of the 
absorbing homogenous states. Note that in this case the rig- 
orous numerical analysis becomes difficult because of the ex- 
tremely low configuration probabilities occurring during the 
oscillations. 



IV. MONTE CARLO SIMULATIONS 

MC simulations are performed on square lattices and on 
RRGs. In both cases the runs are started from random ini- 
tial states. Frequencies and payoffs of all three strategies are 
traced over time. After an appropriate relaxation time the av- 
erage frequencies and payoffs are determined (sampling times 
varied from 10 4 to 10 5 MC steps per site). 
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FIG. 3: Average frequency of defectors (squares), cooperators (dia- 
monds), and loners (triangles) as a function of b for K — 0.1 and 
a = 0.3 on square lattices. The solid (dashed) lines show the stable 
(unstable) stationary values predicted by pair approximation. 



The linear extension of the square lattice with periodic 
boundary conditions was 400 sites for most MC simulations. 
However, in the vicinity of transition points the system size 
was increased to up to 800 sites in order to suppress undesired 
diverging fluctuations. 

The simulations confirm that loners die out for small b < 
6q S( ^ = 1.0262(1). In fact, since loners invade the territory of 
defectors, they can only survive if the frequency of defectors 
exceeds a threshold value. Note that in the vicinity of the 
transition point, the pair approximation predicts slightly lower 
defector frequencies and therefore a higher threshold value. 

The frequency of loners vanishes continuously when de- 
creasing b — ► 6g Sq ' . The numerical analysis of MC data sug- 
gests that this extinction process belongs to the directed per- 
colation universality class [|4 , 35 fRelated critical transitions 
were reported and discussed in Q16[ [28| l. In contrast to 
this non-analytical transition, the pair approximation predicts 
a mean-field type behavior, i.e. the frequency of loners van- 



ishes linearly with b — 6q P ' . 

For b > bg^ the frequency of loners (defectors) increases 
(decreases) monotonously with b. However note that the oscil- 
latory behavior predicted by pair approximation is absent on 
square lattices. The time evolution of the lattice shows cyclic 
invasions for any site but the phases of these local oscilla- 
tions are not synchronized [f3l"|]. Instead, the cyclic invasion 
sustains a self-organizing poly-domain structure. Similar pat- 
terns are observed for spatial rock-scissors-paper games (see 
e.g.||§). 

To illustrate the striking difference between the results ob- 
tained on square lattices and RRG, we compare two trajecto- 
ries determined by MC simulations for the same model pa- 
rameters (see Fig. Q). On square lattices the trajectory evolves 
towards a stable interior fixed point. In contrast, on RRG the 
trajectory approaches a periodic limit cycle. Note that for such 
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FIG. 4: Time evolution of strategy frequencies according to MC sim- 
ulations for b = 1.5, a = 0.3, and K = 0.1. The solid line shows 
the trajectory on a square lattice with 600 x 600 sites. The dashed 
line indicates the limit cycle on a RRG with N = 5 ■ 10 5 sites. 




FIG. 5: Average frequency of defectors versus temptation b obtained 
by MC simulations (closed squares) on RRG and by pair approxi- 
mation (solid line). The minimum and maximum frequency of D is 
indicated by open symbols (simulations) and dashed lines (pair ap- 
proximation). 



large system sizes, the noise amplitude is comparable to the 
line width. The oscillatory behavior is very similar to the pre- 
dictions by pair approximation (see Fig. ||). Moreover, the 
amplitude again increases with b. To demonstrate this nice 
agreement, the average, minimal and maximal frequencies of 
defectors are depicted in Fig. ||. The MC data is obtained by 
averaging over 10 4 MC steps per sites after suitable relaxation 
times for TV = 10 6 sites. In absence of periodic oscillations, 
the difference between the minimum and maximum values 
increases very slowly with b until for b — 6" s = 1.485(1) 
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FIG. 6: Average frequencies of defectors (squares), cooperators (di- 
amonds), and loners (triangles) as a function of b for a = 0.3 and 
K = 0.1 on RRG. The solid and dashed lines show the stable and 
unstable predictions of pair approximation. 

a Hopf-bifurcation occurs. For b > b\ the extremal values 
quickly separate from the average value reflecting the oscil- 
lating behavior in close agreement with pair approximation. 

Simulations for b > frg — 1-82 confirm another predic- 
tion of pair approximation: in finite populations, oscillations 
with increasing amplitudes are eventually stopped by the ex- 
tinction of either one of the strategies and the system ends up 
in the respective homogenous absorbing state. This means if 
cooperators are to die out first, the systems evolves into a state 
with all loners. Obviously any strategy may disappear first but 
the probabilities depend on the parameters, the initial config- 
uration and the system size. For example, about 87% (13%) 
of randomly initialized runs lead to a homogeneous loner (de- 
fector) state for b — 1.8 and TV = 10 6 . 

On RRG the results of MC simulations are well described 
by pair approximation as illustrated in Fig. || For such struc- 
tures, the extinction of loners is expected to be a mean-field 
type process |33|]. Rigorous analysis of our MC data con- 
firms that loners indeed vanish linearly with b — where 
b ( n ' s) = 1.03858(3). 

In the region of coexistence (b^^ < b < b^ gS> ) 
the frequency of loners (defectors) increases (decreases) 
monotonously with b. But note that the cooperator frequency 
in MC simulations slightly increases in the oscillating region 

(6^ 1T8 ' < b < b^^)- The dashed lines in Fig. || represent the 
unstable stationary solution of pair approximation meanwhile 
the symbols are determined by averaging over more than 300 
cycles in the MC simulations. 



V. CONCLUSIONS 

The present investigations confirm that voluntary participa- 
tion in PDG interactions is capable of preventing "the tragedy 
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FIG. 7: Average payoff of cooperators (diamonds) and defectors 
(squares) as well as the average population payoff (circles) a func- 
tion of 6 for if = 0.1 and o — 0.3. The dashed line indicates the 
loners (fixed) income a and the dotted line shows the maximal pay- 
off available for uniform cooperation. Closed (open) symbols refer 
to MC simulations on RRG (square lattice). 

of the commons" and efficiently avoids states of mutual defec- 
tion. Introducing the loner strategy gives rise to a cyclic dom- 
inance that can maintain the co-existence of all three strate- 
gies. With players arranged on a square lattice, the cyclic 
invasions maintain a self-organizing three-color pattern. In 
contrast, different types of oscillations in the frequencies of 
the strategies are observed if players are located on a RRG. 
Both scenarios have the same connectivity z = 4, i.e. the 
players invariably interact with four neighbors. Occasionally 
players reassess their strategy by sampling their neighborhood 
and switch strategy with a probability depending on the payoff 
difference. 



From the view point of sociology (or behavioral sciences) 
the most relevant questions refer to the average individual in- 
come of defectors, cooperators, and loners. Figure [7] clearly 
illustrates that spatial extension promotes cooperation. The 
average payoff of cooperators is significantly higher than that 
of defectors but more importantly, the average population pay- 
off is larger than the loners income a. This means that the 
possibility to participate in the PDG has a positive net return 
for the population. At the same time it remains far below the 
maximum m c = 4 leaving room for further strategical im- 
provements. 

Comparing the results for square lattices and RRG, we note 
that the average payoff of cooperators, and even more pro- 
nounced the average population payoff, turns out to be sig- 
nificantly lower on RRG. Due to the randomly drawn links 
between the players, cluster formation becomes more difficult 
while facilitating exploitation. But recall that in absence of 
spatial structures, only loners survive and forego the chances 
of the PDG. Similarly, for a sufficiently high temptation to de- 
fect (b > 62) the oscillations on RRG usually terminate in the 
absorbing state of all loners (see above). 

In this work we investigate effects of the underlying geom- 
etry on the success of cooperative strategies by comparing the 
game dynamics on square lattices and RRG with equal con- 
nectivity. These two structures are indistinguishable at the 
level of pair approximation. However, MC simulations re- 
veal significant qualitative differences arising from the distinct 
topologies. At the same time, pair approximation turned out 
to reproduce a detailed picture of the MC results obtained on 
RRG. 
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